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THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
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3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 
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Application Papers 
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Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
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DETAILED ACTION 



Response to Amendment 



1. The amendment was received on February 12, 2004 (Paper No. 6B). 

2. The abstract has been received and entered. 

3. The tile has been changed, accordingly. 

4. Proper amendment to the specification has been received (Paper No. 9C) and entered. 

5. Objections made to claims 5 and 15 have been withdrawn. 

6. Applicants' have overcome 35 U.S.C. 1 12 2 nd paragraph rejections and therefore those 
rejections have been withdrawn. 

7. Drawings corrections to Figures 8, 7a, 7b, and 7c have been received. 

Response to Arguments 
1. Applicants' arguments with respect to claims 1-28 (Remarks: pages 8-1 1) have been 
considered but are moot in view of the new ground(s) of rejection. However, upon further 
consideration, a new ground(s) of rejection is made in view of US 4,378,569 (Dallas, Jr. et al.). 
NOTE: Applicants' argue on page 1 1 of remarks regarding claim 20 rejected under 103(a) over 
Meijer in view of Dallas (US 4,378,569). However, applicants' only state "The addition of 
Dallas, Jr. does not overcome the deficiencies of the primary reference (i.e., Meijer) to render 
Applicant's invention obvious." Upon further consideration Dallas, Jr. et al. meets the elements 
recited in applicants' invention. 



2. The information disclosure statement (IDS) submitted on February 05, 2004 (Paper No. 
7) was filed after the mailing date of the First Action but before the mailing date of a Final 



Information Disclosure Statement 
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Action. Accordingly, the information disclosure statement has been considered by the examiner 
and the references are listed on PTO-892 form. 

NOTE: The Search Report (Paper No. 7) was submitted to satisfy the translation requirement for 
reference 1003022 A3 and 2-595-940, which do not have English translation when submitted 
with the IDS filed January 22, 2001 (Paper No. 3). 

Claim Objections 

3. Claim 4 is objected to because of the following informalities: Claim 4 line 2 the word 
"person=s" ought to be - person's — . Appropriate correction is required. 

4. Claim 28 objected to because of the following informalities: The preamble of claim 28 
states "A portable device according to claim 23 . . ." where claim 23 states "A device . . .". Please 
change the preamble of claim 28 to be consistent with the preamble of claim 23. Appropriate 
correction is required. 

5. Claim 18 objected to because of the following informalities: claim 18 line 2 discloses the 
word "comprise" which ought to be — comprises — . Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 
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7. Claims 1-14, 17, 21, 23-24, and 27 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Dallas, Jr. et al. (USPN 4,378,569) (hereinafter, "Dallas"). 

With regard to claim 1 Dallas discloses a method enabling a person to visualize images 
(See, Fig. 1) comprising the steps of: encoding spatial information (encoding video signal 
information by using audio modulation 12 into an acoustical representation. See, col 3 lines 32- 
36) relating to a feature or features contained within an image (i.e., the visual images taken by a 
video camera. See, col. 3 lines 25-32) into the form of one or more polyphonic musical 
sequences (as more clearly seen in Fig. 2 that the video signal generator 10 passes information 
using 64 channels to an audio modulator 12 which outputs (in 64 channels) using 64 modulators. 
These 64 channels are single multi- frequency output and each 64 channels have different tone 
(i.e., polyphonic sequence) as disclosed at col. 3 lines 65 to col. 4 lines 1-4. See, col. 3 lines 45- 
68 to col. 4 lines l-6)(image-to-sound conversion as seen in Fig. 1 and see col. 3 lines 32-42. 
Note: Dallas discloses the signal being of a music at col. 7 lines 5-21); and playing the musical 
sequence or sequence to the person (i.e., playing (outputting) the music sequence to the 
headphones thru which the person listens. See, col. 4 lines 4-6, Fig. 2 element 26. 

With regard to claim 2 it is inherent that the spatial information encoded by selecting a 
note or chord (note: Generally, music inherently includes notes and/or cords) dependent on (a 
distribution) of the feature or features along an axis (image scanning in vertical and horizontal 
directions, col. 3 lines 62-65). 

With regard to claim 3 Dallas discloses the image comprising a letter or a number (Dallas 
discloses broadly "Images" that is visual at col. 4 lines 24-29 and that includes a letter or a 
number). 
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With regard to claim 4 Dallas discloses the image comprising a person's environment 
(Dallas discloses broadly "Images" that is visual at col. 4 lines 24-29 and col. 5 lines 45-49). 

With regard to claim 5 Dallas discloses spatial resolution information by representing the 
image as a 2D image (the images (represented by video frames) at col 3 lines 25-32 and col. 4 
lines 24-29 has the representation of X- Y-axis (64 x 64 matrix), hence 2D image) and forming 
one or more musical sequence, each comprising a series of notes or chords (as described above 
in claim 1), in which i) each note or chord is selected dependent upon the distribution of the 
feature or features along a portion of the 2D image (as described in claim 2 and visual image is 
represented in 64 columns and 64 rows at col. 3 lines 25-32) and ii) different notes or chords in a 
sequence correspond to different portions of the 2D image (See, col. 3 lines 65 to col. 4 lines 1- 
4). 

With regard to claim 6 Dallas discloses the 2D image, which is divided into a matrix of 
pixels (See, col. 3 lines 25-32), and i) each note or chord is selected dependent upon the 
distribution of the feature or features along a column (or rows) of pixels (visual image 
represented by video frame is represented in 64 columns and 64 rows at col. 3 lines 25-32, col. 4 
lines 34-50) and ii) different notes or chords in a sequence correspond to the distribution of the 
feature or features along different columns (or rows) of pixels (See, col. 3 lines 65 to col. 4 lines 
1-4). 

With regard to claim 7 Dallas discloses a different note, which is associated with each 
pixel along a column (pixel in a column from 64 x 64 matrix array or 32 x 32 array, col. 4 lines 
34-37, 41-43) and, if a feature recognized as being present in a pixel, the note corresponding to 
that pixel comprises part of the musical sequence (the output from a video signal 10 representing 
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a pixel is summed and converted into an acoustical signal 12 as seen in Fig. 1. See, col. 3 lines 
32-36 and col. 4 lines 58 to col. 5 lines 1-21). 

With regard to claim 8 Dallas discloses enabling a person to visualize moving features 
comprising the step of playing a plurality of musical sequences corresponding to different 
positions and/or orientations of the moving features (Dallas discloses a video camera in a video 
generator 10, col. 3 line 25, taking a sequence of image of an moving object and converting this 
to an acoustical signal, col. 3 lines 32-36 at different positions and/or orientation at col. 4 lines 
24-30). 

With regard to claim 9 Dallas discloses a subset of a full image that is encoded into the 
musical sequence or sequences (see a subset (each row and column) being encoded in an 
acoustical signal from channels 1-64 as seen in Figs. 1-3). 

With regard to claim 10 Dallas discloses predetermined features that are extracted from 
the image, and said predetermined features are encoded into a musical sequence or sequences (as 
described above in claim 9, each features in each row and column, are predetermined and 
converted into an acoustical signal before summing up to obtain the entire image in an acoustical 
signal as seen in Fig. 1 and also at col. 4 lines 58-66). 

With regard to claim 11 Dallas discloses a feature being simplified by encoding a portion 
of the feature as a musical sequence (each feature (channels 1-64) is being simplified at 101, 103, 
105 as seen in Fig. 5. See, col. 7 lines 55-64). 

With regard to claim 12 Dallas discloses the feature that is encoded by encoding different 
portions of the features as different musical sequences (See, col 3 lines 65 to col. 4 lines 1-4). 
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With regard to claim 13 Dallas discloses the image being encoded into the form of a 
plurality of musical sequences, which are played to the person as a melody (i.e., arrangements of 
sounds) (musical sequence of each feature are being converted in an acoustical signal and played 
to the person thru a headphones. See, col. 3 lines 32-36 and col. 4 lines 4-6). 

With regard to claim 14 Dallas discloses the image being encoded as a plurality of 
musical sequence (as described above), each corresponding to different spatial resolutions 
(different frequencies give different spatial resolutions. See, col. 4 lines 60-62). 

With regard to claim 17 Dallas discloses the notes and chords in the sequence having a 
duration, and the spatial resolution corresponding to a musical sequence being indicated by the 
duration of the notes and chords in the sequence (the listener is informed by a signal indicating 
the duration of the notes and chords between each images in the video frame. See, col. 4 lines 7- 
29). 

With regard to claim 21 Dallas discloses the feature or features having a brightness and 
the musical sequence or sequences having an intensity associated therewith, and wherein the 
brightness of the feature or features being encoded by varying the intensity of the musical 
sequence or sequences (See, col. 4 lines 2-6). 

Claim 23 recites identical features as claim 1 except claim 23 is a device claim. Thus, 
arguments similar to that presented above for claim 1 is equally applicable to claim 23. Note, 
Dallas discloses a device as seen in Fig. 1 . 

With regard to claim 24 Dallas discloses a video camera at col. 3 line 25. 

With regard to claim 27 Dallas discloses the playing means comprising an ear-piece (i.e., 
headphones) at col. 4 lines 4-6 and Fig. 1 element 26. 
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Claim Rejections - 35 USC §103 

8. The following is a quotation of 35 U.S.C 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made, 

9. Claims 22, 25-26, and 28 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Dallas in view of Meijer (USPN 5,097,326). 

With regard to claim 2 2 Dallas discloses video frames of a moving object or a scene 
translating into sound patterns as disclosed above in claim 1. Dallas does not expressly disclose 
image motion cues being translated into equivalent sound patterns, which are utilized to 
segregate foreground features from background in 3D and produce information on relative depth 
between features. Meijer discloses the image motion cues are translated into equivalent sound 
patterns, which are utilized to segregate foreground features from background in 3D and produce 
information on the relative depth between features (See, col. 4 lines 14-21 and the 3D image at 
col. 7 lines 8-12). Dallas and Meijer are combinable because they are from the same field of 
endeavor, i.e., converting an image to sound representation. At the time of the invention, it 
would have been obvious to a person of ordinary skill in the art to combine the teaching of 
Meijer with Dallas. The motivation for doing so is to distinguish between the background and 
foreground so the user can only listen to what is important (i.e., foreground information) and to 
eliminate the information not necessary for the user from the background as suggested by Meijer 
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at col. 4 lines 14-21 by having an example of a 'white line.' Therefore, it would have been 
obvious to combine Meijer with Dallas to obtain the invention as specified in claim 22. 

With regard to claim 25 Meijer discloses a charge coupled detector at col. 13 lines 5-9. 

With regard to claim 26 Meijer discloses the encoding means comprising a 
microprocessor at col. 10 lines 27-37. Note: image processing unit shown in detail here is a 
microprocessor. 

With regard to claim 28 Meijer discloses an imaging means being a portable device at 
col. 2 lines 27-30. 

Allowable Subject Matter 

10. Claims 15-16 and 18-20 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

The closest prior art to Dallas and Meijer are directed to a method and a device enabling 
a person to visualize images as disclosed in independent claims 1 and 23. 

However, the closest prior art fails to disclose anything about the image having a center, 
and the image is being divided into two or more concentric zones, the zone at the center of the 
image being optionally encoded at a higher spatial resolution than the spatial resolution of the 
zone furthest from the center of the image as disclosed in claim 15. Further, the closest prior art 
fails to disclose a colour of the feature or features is being encoded by producing a musical 
sequence or sequences which comprises a plurality of different sets of waveforms mixed in 
variable rations, wherein the different sets of waveforms correspond to different sounding 
instruments having different characteristic harmonic waveforms as disclosed in claim 18. It is 
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for these reasons in combination with all the other elements of the claim that claims 1 5 and 1 8 
would be allowable if rewritten in independent form including all of the limitation of the base 
claim and any intervening claims. Claims 16 and 19-20 are allowable for the same reason as 
claims 15 and 18. 

Conclusion 

1 1 . Applicants amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Shefali D Patel whose telephone number is 703-306-41 82. The 
examiner can normally be reached on M-F 8:00am - 5:00pm (First Friday Off). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo H Boudreau can be reached on 703-305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 




Shefali D Patel 
Examiner 
Art Unit 2621 



May 10, 2004 



